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METHOD AND APPARATUS FOR CREATING RECOMMENDATIONS 
FROM USERS PROFILE BUILT INTERACTIVELY 

Background of the Invention 

This invention relates to entertainment media management systems, 
and. more particularly, to an apparatus and method that recommends a 
program to a user based on program metadata and on user profiles. 

In order to better understand this disclosure, the following definitions 
5 are offered. A program is a media (most likely an electronic media such as 
music, television programs, digital video or a tangible item such as a book, 
compact disc or a live event such as a play, opera, etc.) with validity periods. 
Validity periods define time periods when the program can be delivered to 
users. Program metadata are data that describes a program. 

10 Recently, breakthroughs in digital technologies allow users to receive 

hundreds of video channels. In fact, it is widely believed that the barrier of 
1000 channels will soon be broken. In this new environment, the amount of 
content available is overwhelming. Electronic guides are limited by the 
amount of information that can be displayed on a single screen at a given 

15 time. Additional tools are therefore required in order to help the user find a 
program that he is interested in. One way to do this is to propose recommen- 
dations on programs addressing an individual user's preferences (for example 
user tastes, behavior, demographic information). The prior art includes 
various systems directed toward enhancing television-viewing efficiency by 

20 providing individual suggestions. For example, an early system described in 
US Pat. No. 4,170,782, the content of which is incorporated herein by 
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reference, allows the viewer to preselect a television-viewing schedule of 
desired television channels to be viewed during selected time periods. This 
system also monitors the television viewer and relays this information to a 
central data processing center over a communications link. Other similar 
5 systems, such as those described in US Pats no. 4,264,924 and 4,706,121 , the 
contents of which are incorporated by reference, facilitate access to 
programming but provide no assistance to the user in determining what 
programming to select for subsequent viewing. US Pat. No. 4,745,549 and 
5,075,771, the contents of which are incorporated herein by reference, try to 
10 provide selective access to programming from user preferences built on a 
user questionnaire. However, this kind of interaction based on a general 
questionnaire does not provide any insights as to the desirability of a 
particular program. 

The system described in US Pat. No. 5,223,924, the content of which 
1 5 is incorporated by reference, begins to address the above problem by 

providing a user interface for automatically correlating the user preferences 
with TV program information and creating and displaying a personalized 
user program database based on the results of the correlation using a "free- 
text search" engine. In this system, the user profile is limited to automatically 
20 update the database of "preferred" programs using direct user feedback. 

US Pat. No. 5,410,344, the content of which is herein incorporated by 
reference, provides a system for selecting programs for presentation to a 
viewer. The programs selected from a viewer preference file built with 
explicit ratings on program metadata and from a neural network engine. A 
25 neural network is an black box of processing elements with a limited number 
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of inputs and outputs. These processing elements are able to "learn" by 
receiving weighted inputs that, with time and repetition, can be made to 
produce appropriate outputs. The viewer preference file is accessible only 
locally on the viewer terminal and contains no implicit ratings so that the 

5 value and efficiency of recommendations are limited- 
US Pat. No. 5,534,91 1, the content of which is incorporated by 
reference, describes a virtual personal channel but user profile is stored 
locally and is non persistent between sessions. In U.S. Pat. No. 5,801,747, 
the content of which is incorporated by reference, a user-profiling server is 

10 described. However, user profiles are built only based on viewing activities 
of the user. Further, the system can only display preferred channels or 
categories. 

Some prior art systems address the problem of suggesting programs 
from a large program database such as a movie database (for video on 

15 demand) as described in U.S. Pat. No. 5,483,278, the content of which is 
incorporated by reference. In this system, the user makes explicit ratings on 
program metadata. The system presents programs in an individual order and 
the user can rate them explicitly (e.g., by selecting "liked" or "disliked") and 
the order of program presentation to the user is modified. The user profile is 

20 stored locally with respect to the terminal and is non persistent so that the 
interest of such a system is limited. 

Referring to US Pat. No. 5,758,257 and US Pat. No 6,020,883(the 
content of which is incorporated by reference), two profiles are built from 
explicit ratings, one on content data and one on viewer's preferences. An 
25 "agreement matrix" is then calculated by comparing the two profiles and a 
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"virtual channel" is generated, providing personal recommendations on 
programs to each view^er. A feedback path with monitoring information is 
also provided so that user profiles and content profiles can be improved. This 
system works also with movies, music or book databases (kiosk). This 
5 system includes an "agreement matrix" and is well adapted to work inside a 
powerftil terminal but not on a profiling server with millions of users and a 
limited resource terminal. 

In US Pat. No. 5,798,785, the content of which is incorporated herein 
by reference, a terminal offers the ability to assist a viewer in choosing 

0 programs for viewing. Program recommendations are built from information 
on frequently viewed programs, persistent profile information and user mood 
information. This information is interpreted into preferred program indicators 
and matched with the program metadata to suggest a list of programs. The 
first limitation of this system is that it is not able to make suggestions by 

5 correlated profiles of different users. The second limitation is that the data 
gathering and analysis techniques used are adequate for a powerful terminal 
but not for a central profiling server with millions of users. 

Therefore, what is needed is a system and method which overcomes 
the drawbacks of the prior art, generating user program recommendations 
0 from the knowledge of the user profile itself, and also fi-om the profiles of its 
neighbors that have the same tastes. 
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Summary of the Invention 

A computer-readable medium is encoded with a method of profiling 
that collects explicit and implicit data on users related to programs that can 
be accessed by users and to provide program recommendations to each user. 

^ The system and method is designed for the server-side and limited 

resource terminals to set up user profiles from explicit and implicit data 
simultaneously on a full program schedule database as well as a specific 
program (e.g.. movie) database. 

Schedules on program availability and program metadata are 

10 broadca^it \o or received on request by the user terminal through a 

communication medium from a central system that contains a network of 
physical scr\ lts (network, application, database server for example). When, 
from a terminal, a user interactively rates a program or a field of the program 
metadata, this explicit rating is sent to the profiling server through the return 

15 path and is stored in the user's profile database. The profiling server collects 
information on user tastes with respect to program metadata. The profiling 
ser\ er also receives implicit information from user behavior such as user 
program remind actions, user purchase actions, or user monitoring 
information. All these implicit ratings complete the profile of each user. 

20 Then the recommendation engine of the profiling server evaluates the profile 
of users with the schedule of programs that are now available and that will be 
available in the near future. Based on the evaluation process, the recom- 
mendation engine builds recommendations of programs for users. The 
recommendation engine works with several different filtering engines such 

25 as a content filtering engine and a collaborative filtering engine and treats 



5 

BNSOOCia <WO 0115*49At_L> 



wo 01/15449 



PCT/IB0O/On31 



one user at a time in each filtering engine. Recommendations are weighted 
and gathered at the output of these filtering engines and a unique 
recommendation per program is generated by the recommendation engine for 
each user. Then the list of program recommendations is stored in the profile 
5 of the user. These recommendations are sent to each user terminal on 

terminal request or by a push from the profiling server. The user can access 
the resulting recommendations through a user interface that displays 
recommendations of the best programs for him with different scenarios such 
as portal, menus, tickers, virtual channels, and lists. 

io An object of the invention is to provide a system and method for 

server-side and limited resource terminals to set up user profiles from 
explicit and implicit data simultaneously on a full program schedule database 
as well as a specific program database. Another object of the invention is to 
generate user program reconmiendations from the knowledge of the user 

1 5 profile itself, and also from the profiles of its neighbors that have the same 
tastes. 

Brief Description of the Drawings 

The features and advantages of the present invention will become more 
apparent and understood from the following detailed description which should be 
20 read in conjunction with the accompanying drawings, in which: 

FIG. 1 is a block diagram of a profiling and recommendation system; 
FIG. 2 is a schematic diagram of a terminal screen of the invention* 
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FIG. 3 is a flow chart of the method of collecting the explicit ratings 
of the user on program metadata; 

FIG- 4 is a flow chart of the method of collecting implicit data on the 
programs the user has watched by storing the switches between channels; 

5 FIG. 5 is a flow chart of the method of collecting implicit data on the 

programs the user has watched by evaluating the user's viewing time; 

FIG. 6 is a block diagram of a system of the invention including a 
two-way profile server that receives, stores program metadata and user 
explicit/implicit data and then builds and returns program recommendations; 

10 FIG. 7 is a block diagram of a recommendation engine that provides 

program recommendations from a set of filtering engines; 

FIG. 8 is a flow diagram of a method for selecting programs from the 
user profile and from program metadata in the content filtering engine; 

FIG. 9 is a block diagram of a user terminal including network 
15 connections and user interactions; 

FIG. 10 is a representative program description guide with explicit ratings 
displayed on top of a TV program. 

Detailed Description of the Preferred Embodiment 

The following embodiments of the present invention will now be 
20 described in detail relative to Figures 1 to 10. 

Referring now to Fig. 1 , the present invention 100 is shown as part of an 
expanded program delivery system 100 that delivers programs 102 and program 
metadata 104 from a central server 106 to a user terminal 108 through a network 
110. In the case of a cable television system, the central server 106 can be located 
25 at the operator head-end and the user terminal 108 can be an operator set-top box 
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10 



(STB) or the television itself. In the case of an Internet solution, the central server 
106 can be integrated into the web and application servers and the terminal 108 can 
be a personal computer (PC) 114. The terminal 108 is a device that displays 
programs 102 and program metadata 104 to a user and that can interact with the 
user. In the television field, the terminal 108 is usually a set-top box (STB) or the 
television set. The terminal 108 can also be a wireless device (PDAs, mobile 
phones, etc.). 

Major System Components 

Referring again to Fig. 1 , the system 100 operates within the context 
of a content or service provider system for communicating programs 102 and 
programs metadata 104 to users 116. This system 100 is composed of two 
parts 118 and 120. The first part 118 is the central server 106 that manages 
and delivers content to the users 116. The second part 120 is the user 
terminal 108 that receives the content and allows the user 116 to interact with 
the central server 106. A content management system 122 manages programs 
102 and programs metadata 104 and is synchronized with the content 
delivery system 124 that sends programs 102 to the network 110, destined to 
the users 116. The content management system 122 also sends program 
metadata 104 to the profiling server 126. The profiling server 126 collects 
20 information on programs metadata 104 and on user profiles and stores these 
data in the program metadata database 128 and in the users profile database 
130. The profiling server 126 is the main component of the invention: from 
the collected data, the server 126 builds and delivers program recom- 
mendations to each user 116. The recommendations are received by the 
terminal 108. A user 116 can use different terminals 108 to access its profile 
on the profiling server 126. The profiling server 126 imposes no constraints 
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on the infrastructure of the network 110 if we assume a reasonable 
bandwidth adequate for the number of users 116. 

It should be noted that program metadata 104 may also be obtained 
from other sources, such as a ftp xml file or in the media itself, digitally 
5 embedded in the content and need not come from the content management 
system 122. 

Program Metadata and User Profiles 

A program 102 is described by its metadata 104. Metadata 104 of 
programs contain a large set of possible fields but only a few subsets of this 
10 fields are useful for the profiling server 126 and its recommendation engine 
132. A more detailed description of a preferred embodiment is provided as 
follows. 

(a) Program ID 218: The Program ID 218 uniquely identifies a 
particular program 102. The ID 218 can be an integer (32 bits) or a long 

15 integer (64 bits) or a string of bytes not exceeding 2000 bytes. 

(b) Title 208: The title 208 uniquely identifies the main title of the 
program 102 and is defined as a string. 

(c) Category 210: The category 210 identifies the media content and 
media type category of a program 102. The media type category defines the 

20 type of program 102 such as a movie, song, a documentary, a live event, a 
software, etc. The media type category is defined in one level, which means 
it is one element of a list of category values. The media content category 
indicates the nature of the content of the program 102. It consists of three 
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hierarchical level categories. For example, a program 102 might have the 
media content level categories of (1) sport, (2) football, and (3) Arsenal. In 
the case of an Electronic Program Guide (EPG), the two higher-level fields 
can be compatible with category fields defined in the Digital Video 
Broadcasting Standard (DVB). Category fields are defined as strings. 
Optionally, a unique ID can be added to the string definition. 

(d) Description 220: The description 220 describes the program 102 in 
a textual form and is defined as a string. This field is optional for the 
recommendation engine but useful for the users 116. 

(e) People 212: People 212 identify the stars or other persons of 
interest in the program 102. One individual is defined by a people category 
and by name. Examples of people category are: actor, producer, presenter, 
famous, etc. People categories and people names are designed as strings. 
Optionally, a unique ID can be added to the string definition. 

(f) Keywords 214: Keywords 214 describe the program 102 by a list 
of keywords. Each keyword represents the content of one part of the program 
102. These keywords can be interpreted without the category fields. For 
example, a keyword might be the "Louvre" in the context of the category 
"documentary, travel, France". Optionally, a unique ID can be added to the 
string definition of the keyword. 

(g) Parental rating 216: Parental rating 216 identifies restrictions 
regarding who can access the associated program 102. For example, the 
rating might be "users older than 13". Classification can be by age or based 
upon content such as violence, language, nudity and/or sex. Parental control 
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attributes are defined as strings. Optionally, a unique ID can be added to the 
string definition. 

User profiles are built on explicit and implicit data 134, 136 collected 
by the profiling server 126. Explicit data 136 is data that are produced by a 

5 direct interaction with the user 116. Each user 116 can indicate using a rating 
scale his preferences on or his reaction to a specific program 102 or a 
program category or people or keywords. All the user ratings using the rating 
scale are collected by the profiling server 126 and are stored in the users 
profile database 130. When a user 116 purchases a program 102 or makes a 

10 remind action on a program, these actions are also stored in the user's profile 
database 130. Implicit data 134 is data that is monitored by the terminal 108 
and sent to the profiling server 126. This data 134 is also stored in the user's 
profile database 130. Implicit data 134 gives information on the viewing 
habits of the user 116, such as which programs 102 he has watched or used, 

15 and the time of day the program was viewed. 

The user profile database 130 is structured as follows: 

(a) Basic data: When users buy or log on to the terminal 108, he must 
identify himself. The profiling server 126 collects this information. Examples 
of this information are: nickname, age, zip code, gender, income, etc. 

20 (b) History data: This data contains all the explicit and implicit data 

134 and 136 collected. 

(c) Processed data: This data contains information that is generated by 
the profiling server 126. For example, this includes program 
recommendations data generated for each user. 

11 
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Referring now to FIGs. 2 and 3, a submethod 300 is described that 
acquires explicit data 136 by collecting the user's tastes by means of a rating 
process. In a first step 302, the terminal 108 (shown in FIG 2) receives 
program metadata 104 and stores the data locally. In a second step 304, the 
metadata is displayed when necessary or requested. In a third step 306, 
following the display of the metadata 104, the user can select a field 214 of a 
displayed program 200, assign a grade 206 for that field 214 and validate this 
rating using a validate button such as that on a remote control or touch 
screen. In a fourth step 308, the rating 206 is saved locally on the terminal 
108. In a fifth step 310, the rating is sent to the profiling server 126 
asynchronously. A delay can be added before the transmission of the 
profiling server 126 in order to facilitate the management of the load of the 
server. When the delay is set to zero, optionally, the profiling server 126 can 
activate the recommendation engine 132 and immediately returns program 
recommendations 204 generated fi-om the new rating. Fields 222 that can be 
rated are the metadata 104 available for each program 200 such as title, 
categories, people, keywords, etc. The same rating process also applies to 
purchases and reminder requests acquisition. 

Referring now to FIG. 4 , a submethod 400 is described for 
acquisition of implicit data 134 on a terminal 108. In a first step 402, a time 
counter is initialized and the system is waiting a user action. In a second step 
404, the user changes a channel. In a third step 406, an evaluation of the 
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manner in which channels are changed (the "channel zapping") is performed. 
This evaluation determines the time spent on the previous chaimel. In a 
fourth step 408, the duration between zapping is evaluated. In a fifth step 
410, if the duration between zapping is greater than the limit, zapping 

5 information is accepted and is saved locally 120. In a sixth step 412, the 
method collects a predefined (configurable) number of channel 
changes/zappings on the terminal 108. In a seventh step 414, the method 
sends this information to the profiling server 126 asynchronously. To use 
implicit data 134, the recommendation engine 132 does not only need to 

10 know zapping information but also how long a user has watched a program 
200. This information can be buih from the zapping information and from 
the program schedule on the server side 106. However, this additional 
information can also be built up in the terminal 108. 

Referring now to FIG. 5, this last option has the advantage of reducing 
15 the load of the server 106. The initial steps 502, 504, 506 of this submethod 
500 are the same as that of FIG. 4 and this submethod works in parallel with 
the main method shown in FIG. 4. After the channel zapping evaluation 506, 
an evaluation step 508 is added which evaluates the programs 200 watched. 
This evaluation step 508 determines how long the current program 200 of the 
20 previous channel has been watched by the user. In a time check step 510, if 
the program 200 is terminated, the time the program was watched is 
evaluated to determine whether it exceeds the predetermined limit (which is 
configurable). If so, in a saving step 512, the program-watched information is 
stored locally on the terminal 108. In a collection step 514, the method 
25 returns to the first step 502 until the number of program 200 watched reaches 
a maximum. In a subsequent step 516, after having collected a predefined 

13 



BNSDOCID: <WO 0n5449Al_L= 



(configurable) number of watched program information on the terminal 108, 
the information is sent to the profiling server 126 asynchronously. 

Profiling System 

Referring now to FIG. 6, a block diagram of the preferred 
5 embodiment of the profiling server 126 is described. The profiling server 126 
includes a core platform system 602 that synchronizes all the subsystems 
(132,604,606) of the profiling server 126. First, a program agent system 604 
receives program metadata 104 from the content management system 122, 
parses the program metadata 104, converts the program metadata into an 

10 internal format as described earlier, and stores the new program metadata in 
the program metadata database 128. Only the program agent system 604 has 
to be adapted when a different content management system 122 is connected 
to the profiling server 126. In a matter similar to the program agent system 
604, a user agent system 606 is the gateway between the user terminal 120 

1 5 and the users profile database 130. The user agent system 606 receives 
explicit and implicit data 136 and 134 respectively from the user terminal 
120. It identifies, validates and then stores the data in the user profile 
database 130. If the terminal 120 requests immediate program 
recommendations built on the newly received data, the user agent system 606 

20 sends the request to the core platform system 602 that activates the 

recommendation engine 132. The recommendation engine 132 returns the 
program recommendation list to the core platform system 602 that sends the 
recommendations to the terminal 120 through the user agent system 606. The 
core platform system 606 controls and manages program and user agent sys- 
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terns 604 and 606, respectively, as well as the recommendation engine 132. 
The profiling system 126 has been designed as a flexible architecture. 

Recommendations Engine 

5 Referring now to FIG. 7, a block diagram of the recommendation 

engine 132 is shown. The engine 132 has two working modes. In a first 
working mode, called a low priority batch mode, the recommendation engine 
132 builds new program recommendations for each user when the core 
platform system 602 informs it that the new program metadata 104 has been 

10 added to the program metadata database 128. Depending on the size of the 
user's profile database 130, the update of each user can take a long time. A 
second mode with a high priority allows the profiling system 600 to respond 
faster to a request of a user. In this mode, the profiling server 126 brings 
instant gratification of the user by giving him program recommendations 

15 related to the new rating he has sent to the profiling server 126. By 

evaluating one user at a time, the recommendation engine 132 can easily be 
adapted to multi-processors and multi-servers architecture with hash coding 
techniques on users. 

In a first substep 702, the recommendation engine 132 selects the user 
20 it wishes to treat. In a second substep 704, once the user is selected, the 

recommendation engine 132 reads the user profile stored in the users profile 
database 130. Then the recommendation engine 132 activates different 
filtering engines 706, 708 and 710 with that profile. The recommendation 
engine 132 supports multiple algorithms and can aggregate the result of the 

15 



BNSCOCIO: <WO 0n5449AlJ_> 



wo 01/15449 



PCT/IBOO/01131 



am 



10 



20 



25 



different filtering engines 706, 708 and 710 to a single list of prograr 
recommendations. In a third substep 720, the user profile in the user profile 
database 130 is updated. In order to accomplish this, the weighting 
coefficients 712, 714 and 716 are added to each respective program recom- 
mendation of each filtering engine 706, 708 and 710, Coefficients 712, 714 
and 716 differ depending on the filtering engine 706, 708 or 710 to which 
they apply and also depending on the program content category because a 
filtering engine can be better adapted to one content category than another 
content category. For example, the collaborative filtering engine 708 gives 
better results with the movie content category than with the sport content 
category. 



can 



The coefficients 712, 714 and 716 can be tuned manually. They - 
also evolve dynamically by using a feedback adaptive algorithm well adapted 
to heuristic algorithms. The recommendation engine 132 adapts the 
15 coefficients 712, 714, and 716 by using for all users either an evaluation of 
the difference between the recommendation grade and the grade the user has 
given to that program or an evaluation of the difference between the 
recommendation grade and the duration the program 200 has been watched 
by the user. By default, the recommendation engine 132 uses two 
complementary filtering algorithms: content filtering and collaborative 
filtering. The content filtering engine 706 generates recommendations 
separately for each user while the collaborative filtering engine 708 finds 
correlation between the user tastes. The collaborative filtering requires 
bootstrap to start to generate recommendations while the content filtering 
does not. The content of the filtering engine 706 can generate program 
recommendations as soon as at least one rating has been made by the user 
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116. Of course, if only a few ratings are given, the customization of the 
programming recommendations is low. 

One solution consists of providing a default profile when the user 
profile is first created. This default profile can be built on the basic data 

5 mentioned under the heading "Program Metadata and User Profiles", 

available on the user 116 and on aggregated data such as the best or the most 
rated programs by other users. Another more interesting solution is to leave 
the choice to the user 116 to choose, as a default profile, the profile of a 
famous person he likes. Collaborative filtering has proven useful for movie, 

10 b<K)k. music or documentaries. Content filtering has powerful availability 
with ncu s. sport, people, and all strongly categorized items. In the TV con- 
text it also fits for periodic programs 102. The collaborative filtering engine 
708 requires more intensive CPU resources than the optimized content 
filtering engine 706, the collaborative filtering engine 708 can only be used 

15 for a subset of all the programs 102 (e.g. movies, documentaries, music). 
These two algorithms respectively have proved to be well adapted to item 
recommendation and are complementary. Nevertheless, the recommendation 
engine 132 can easily integrate another filtering engine such as a more 
general social filtering engine 710. Each filtering engine 706, 708, 710 has 

20 access to the program metadata database 128 with the explicit and implicit 
data 136 and 134. 

FIG. 8 describes the internal architecture of the content filtering 
engine 706. custom developed according to the needs of the present 
invention. The engine 706 is optimized for structured and categorized 
25 metadata. When the content filtering engine 706 is started, program metadata 
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104 with a right validity period are selected in a selection step 810 and 
indexed in an indexing step 812 for each program field such as "title", 
"people", "content category", and "keyword" used by the engine. From the 
user profile already loaded 704, the engine 706 separates infonnation on user 
tastes 802, user purchases 804, user reminder requests 806 and use 
monitoring information 808. There is no technical limitation to the number of 
subjects for possible characterization or categorization -other user 
information can be used by the engine 706. Each information category 702, 
704, 706 and 708 goes through a tuned matching engine for tastes 814 for 
purchases 816 for reminder requests 818 and for monitoring 820. Each 
matching engine 814, 816, 818 and 820 uses the same algorithm based on 
static matching rules and non-linear distance estimation between program 
metadata 104 and user profile fields (defined in user profiles section) but 
with different rules and distance estimation coefficients for each matching 
engine. Rules and distance estimation coefficients are based on heuristics 
built on the knowledge acquired through development, study, and 
experimentation. Use of matching rules allows the purveyor to find a balance 
between search (Indexes) and knowledge (Rules), the two main approaches 
of AI. Details on rule-based algorithm can be found in Charies Forgy, "The 
Rete Algorithm: A Fast Algorithm for the Many Pattern / Many Object 
Pattern Matching Problem", 1979, PhD thesis, Carnegie-Mellon University, 
the content of which is incorporated herein by reference. Due to the indexing 
techniques (read access in 0(1)) used on program metadata 104, the content 
filtering engine 706 is very fast and can support a large number of users 116. 
The bottleneck is on database access and not on the engine 706. Structured 
program metadata 104 is key to guaranteeing the satisfaction of program 
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recommendations 204 and to guaranteeing a fast content filtering engine 706. 
Programs suggested by the matching engines 814, 816, 818 and 820 are then 
weighted using a weight coefficient 822, 824, 826 and 828 respectively and 
aggregated in a list of program recommendations. A first level 830 

5 aggregates program recommendations built from the user behavior (pur- 
chase, remind, monitoring) and a second level 832 aggregates these 
recommendations with the recommendations built in the taste matching 
engine 814. The matching engines (814, 816, 818 and 820) take into account 
at what time the program 200 has been watched, reminded and/or purchased. 

10 For example, some users 116 watch movies in the evening and news in the 
morning while others watch news in the evening. The recommendation 
engine 132 tries to follow this time pattern when it generates the program 
recommendations. The content filtering engine 706 generates as output a pro- 
gram recommendation list 906 with a grading 920 on the programs of the list 

15 918. 



User Terminal 

Now referring to FIG. 9, the data flow of the terminal 120 is shown. 
The terminal 120 is a device that displays programs 102 and program 
metadata 104 to a user 116, collects and sends explicit and implicit data 136 
20 and 134, respectively, on users to the profiling server 126. A user 116 

interacts with the profiling server 126 through the terminal 120 that displays 
its personal recommendations 204. 

Now referring to FIG. 10, a real case of the process of interaction with 
the user 116 is shown. The terminal 120 displays a program description panel 
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902 with information on one program on top of another. Metadata 104 is 
displayed, including the program title 908, the content category 910 and a list 
of people 912. The rating panel 904 applied on the selected item 914 ("Peter 
Barton- in this example) and the user 116 can change the grade of the 
explicit rating 916. Program recommendations available for the current user 
116 are displayed in the recommendation panel 906. Each program 
recommendation is represented by its title 918 and its grade 920, in this case, 
one to five stars. Different processes of interaction can also be used. For 
example, the program recommendations 906 can be displayed in a portal, on 
menus or he received by e-mail. 

Muliipic variations and modifications are possible in the embodiments 
of the invention described here. Although certain illustrative embodiments of 
the in\cntion have been shown and described here, a wide range of 
modifications, changes, and substitutions is contemplated in the foregoing 
disclosure. In some instances, some features of the present invention may be 
employed without a corresponding use of the other features. Accordingly, it 
is appropriate that the foregoing description be construed broadly and 
understood as being given by way of illustration and example only, the spirit 
and scope of the invention being limited only by the appended claims. 
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What is claimed is: 



1 1. A system for generating program recommendations for a plurality of users 

2 based on program metadata and user profiles, the system comprising a 

3 central profiling server unit, the unit comprising: 

4 (a) means for importing program metadata from a program metadata 

5 source; 

6 (b) means for storing the program metadata into a program metadata 

7 database; 

8 (c) means for collecting explicit and/or implicit information on program 

9 metadata from user interaction therewith; 

10 (d) means for storing the explicit and/or implicit information into a user 

1 1 profile database; 

12 (e) means for building program recommendations for users based on 

13 program metadata and user profiles, through use of a plurality of filtering 

14 engines for evaluating user profiles with a schedule of available 

15 programs; 

16 (f) means for gathering and weighting the program recommendations in 

17 order to generate a list of program recommendations per user; £ind 

18 (g) means for transmitting a list of program recommendations for users to 

19 user terminals for display, 

1 2. The system of claim 1 , further comprising means for storing said list of 

2 program recommendations in a user profile database. 
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1 3. The system of claim 1 , wherein the means for importing program metadata 

2 from a content management system and the means for storing the program 

3 metadata into a program metadata database are comprised in a program agent 

4 system. 

1 4. The system of claim 1 , wherein the means for collecting explicit and/or 

2 implicit information on program metadata and users, the means for storing 

3 the explicit and/or implicit information into a user profile database, and the 

4 means for transmitting a list of program recommendations for users to user 

5 terminals are comprised in a user agent system. 

1 5. The system of claim 1, wherein the means for building program 

2 recommendations, the means for gathering and weighting the built program 

3 recommendations, and the means for storing said list of program 

4 recommendations in a user profile database are comprised in a 

5 recommendation engine. 

1 6. The system of claim 1, wherein the program metadata source is a content 

2 management system. 

1 7. The system of claim 1 , further comprising a terminal device having; 

2 (a) means for receiving program metadata and user program 

3 recommendations; 

4 (b) means for displaying the program metadata and user program 

5 recommendations; and 

6 (c) means for collecting and sending explicit and/or implicit information 

7 on program metadata and users to the central profiling server. 
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1 8. The system of claim 7, further comprising means for storing the program 

2 metadata and user program recommendations for later access. 

1 9. A computerized method encoded on a computerized medium, the method 

2 providing program recommendations to a user based on program metadata 

3 and user profile and comprising the steps of : 

4 (a) accessing a server-side program metadata database; 

5 (b) presenting the program metadata on a user terminal for user selection 

6 and rating; 

7 (c) collecting explicit and/or implicit information on program metadata 

8 and the user; 

9 (d) creating at least one user profile based on collected user explicit 

10 and/or implicit information; 

n (e) comparing the program metadata with the user profile for building 

12 program recommendations using (i) a recommendation engine supporting a 

13 plurality of filtering engines at the same time, generating a program 

14 recommendation list associated with each filtering engine and the user; (ii) a 

15 gathering and weighting engine, applied to the filtering engines output, for 

16 generating a list of program recommendations per user; 

17 (f) transmitting the list of program recommendations to the user terminal; 

18 and 

*9 (g) presenting the list of program recommendations to the user. 

1 10. The method of claim 9, fijrther comprising the step of storing on the 

2 server side the list of program recommendations. 
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11. The method of claim 9, further comprising the step of : 

(a) receiving on terminal-side program metadata and user program 
recommendations; 

(b) displaying on terminal- side the program metadata and user program 
recommendations; and 

(c) collecting and sending from terminal explicit and/or implicit 
information on program metadata and users to the central profiling 
server. 

12. The method of claim 9, wherein filtering engines include a content 
filtering engine using indexing techniques on program metadata and 
matching engines based on static matching rules and non-linear distance 
estimation between program metadata and the user profile. 



24 



wo 01/15449 



PCT/lBOO/01131 




wo 01/15449 



PCT/IB00/0n3I 



2/10 




S^'BSTfTUTE SHEET (RUL£ 26) 



BNSDOCID; <WO 011S449AlJ_; 



wo 01A5449 



PCT/I BOO/01 131 



3/10 




BNSDOCIO. <WO 01 15449A1J_> 



wo 01/15449 



PCT/IB00/0n31 



4/10 






BNSDOCID:<WO 0115449A1 I > 



wo 01/15449 



PCT/IBOO/01131 



5/10 





BNSDOCID; <WO 0115449A1.1 



wo 01/15449 



PCT/IBOO/01131 



6/10 




wo 01/15449 



PCT/IBOO/01131 



7/10 




BNSDOCID: <WO OUS4ZSA1 I > 



8/10 




wo 01/15449 



PCT/IBOO/01131 



9/10 





BNSD0CID:<W0 011S449A1 I > 



INTERNATIONAL SEARCH REPORT 



Inte tonal Afif>UcaUon No 

PCT/IB 00/01131 



A, CLASSIRCATION OF^SUBJECT MATTER 

IPC 7 H04N7/173 H04N7/16 



According to International Patent Classification (IPC) of to both national dassificaticin and IPC 



B. FIELDS SEARCHED 



Mntmum documentation searctved (classificabon system toUowed by classification symbols) 

IPC 7 H04N 



Documentation searched otfier ttian minimum documentation to the extent that such documents are tnctuded in ihe fields searched 



Bectronic data base consulted during (he international search {name of data base and, wfiere practical, search temts used) 

EPO-Internal . WPI Data, PAJ 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Categofy " Citation ai document, with indication, whiere appropriate, of the relevant passages 



Relevant to deum No. 



US 5 758 257 A (HERZ FREDERICK ET AL) 
26 May 1998 (1998-05-26) 

column 11, line 45 -column 12, line 6 
column 13, line 29 - line 54 
column 19, line 6 - line 13 
column 34, line 12 - line 23 
column 40, line 11 - line 20 

-/- 



1,2,7-12 
3-6 



m 



Further documents are listed in the continuation of box C. 



0 



Patent family members are listed in annex. 



* Spedal categories of cited documents : 

'A* document defining the general state of the art which is not 

cortsidered to be of particular relevance 
*E* eailter document but putJfished on or after the international 

filing date 

'L* document which may thrcmr doubts on priority claim(s) or 
which is cited to establish the pubticabon date of arwther 
citation or other special reason (as specified) 

'O' document referring to an oral cfisdosure. use. extitbition or 
other means 

'P' document puUistied prior to the international ffling date but 
later than the priority date claimed 



later document publisfied after tfie international fiiirig date 
or priority date and not in conf&ct with ttie appticaton but 
cited to understand the princifrie or theory undedying tfra 
invention 

'X* document of particular relevance: ttie claimed invention 
carwKM be considered novel or cannot be considered to 
involve an inventive step when the document is taken alone 

*Y* documerrt of particular relevance: the claimed invention 

cannot be considered to involve an inventive step when the 
document is combined with one or more O&ter such docu- 
ments, such combination being obvious to a person sktQed 
in the art 

'&' document member of the same patent family 



Date of the acOjal completion of the intemational search 



13 October 2000 



Date of mailing of the intemational search report 



20/10/2000 



Name and maUing address of (he ISA 

European Patent Office. P.B. 561 Q Patentlaan 2 
NL - 2280 HV Rijswipt 
Tel. (+31-70) 340-2040, Tx. 31 651 epo nl. 
Fax: (431-70) 34<«01 6 



Authorized officer 



Sindic, G 



Forni PCT/lSA/210 {»ecand st»«> (July 1 992) 
. <WO 01 1 S449A 1_L> 



page 1 of 2 



INTERNATIONAL SEARCH REPORT 



inU Jonal Application No 

PCT/IB 00/01131 



C.{Contlnuatk)n> DOCUMEffTS CONSIDERED TO BE RELEVANT 



Category • Citation ol document, with indication,where appropriate, oJ the relevant passages 



Relevant to claim No. 



WITTIG H ET AL: "INTELLIGENT MEDIA AGENTS 
IN INTERACTIVE TELEVISION SYSTEMS" 
PROCEEDINGS OF THE INTERNATIONAL 
CONFERENCE ON MULTIMEDIA COMPUTING AND 
SYSTEMS, US, LOS ALAMITOS, CA, 

15 Hay 1995 (1995-05-15), pages 182-189, 
XP000603484 

page 183, left-hand column, paragraph 2 
page 184, right-hand column, paragraph 2 
page 185, right-hand column, paragraph 3 
-page 187, right-hand column, paragraph 4 

NYGREN K ET AL: "An Agent System For 
Media On Demand Services" 
PROCEEDINGS OF THE INTERNATIONAL 
CONFERENCE ON THE PRACTICAL APPLICATION OF 
INTELLIGENT AGENTS AND MULTI-AGENT 
TECHNOLOGY, 

22 April 1996 (1996-04-22), pages 
437-453, XP002086093 



1,4-6,9 



3-6 



page 440, 

page 441, 

page 443, 

page 445, 

page 446, 

page 447, 

page 449, 



paragraph 1 
paragraph 6 
paragraph 
paragraph 
paragraph 
paragraph 
paragraph 



paragraph 2 



paragraph 5 



- paragraph 4 



US 5 798 785 A (BONNER ALFRED E ET AL) 
25 August 1998 (1998-08-25) 
column 29, line 26 -column 30, line 39 
column 32, line 54 - line 62 



1,9 



Form PCT/I5A/S10 (oontinuatran at saoond shMt) (Jiiy 1 992) 
) 0115449A1_I_> 



page 2 of 



2 



INTERNATIONAL SEARCH REPORT 

Information on patent famfty members 



tnt« donal Application No 

PCT/IB 00/01131 



Patent cJocument 


Publication 




Patent f amay 




Pubticatton 


cited in search report 


date 




member(s) 




date 


US 5758257 A 


26-05-1998 


All 




D 
D 


0 c n 'J 1 00a 

tb— Uo-iyyy 




Alt 
AU 




A 

A 


ly— oo-iyyo 




r A 




A 

A 


uo— oo-iyyo 




rp 

Lr 


U/ 70300 


A 
A 


Oyf AO 1007 




11^ 


OU^Uooo 


A 

A 


ni no onnn 

ui— u^-zoou 




WO 


9617467 


A 


06-06-1996 




us 


5734720 


A 


31-03-1998 




lie 




A 

A 


1 0 nc 1 nno 
iy-0b-199o 






[:7c jinoQ 


A 

A 


1 0 AC 1 AOO 

ly-ob-iyso 




lie 




A 

A 


in 11 1 nno 
10-11-1998 




lie 


OUOO/ 


A 

A 


1 1 n7 onnn 
11-U/-ZUU0 




lie 




A 

A 


00 no OAAA 


US 5798785 A 


25-08-1998 


All 

AU 


1 '3'3'570C 


A 

A 


19-0o-199b 




Ail 
AU 


AO'S 1 A Q 


D 
b 


oc AC 1 nno 
Zb-0o-199o 




Ail 
AU 




A 

A 


1 0 AC 1 Anc 

iy-U0-199b 




DK 




A 

A 


oc no 1 AA7 

ZO— 00-199/ 




r A 


0 1 771 C 0 


A 

A 


AO f\£. 1 nr\ c 

0o-0o-199b 




r A 
tA 


771 C/l 


A 

A 


AO AC 1 Anc 

□o-0D-i99b 




CP 

Lr 


A70ono7 


A 


1 0 AO 1 AAC 

1 0-09-1995 




C P 

tr 


U/o^U Jl 


A 

A 


1 0 Art i Arte 

1o-09-199d 




CP 

tr 


uyoji 10 


A 

A 


AO 1 0 1 AAA 

Oo-lZ-1999 . 




IP 




1 


1 7 AC 1 Am 

1/-0O-199/ 




IP 

Jr 




1 


^ A 1 A 1 AOT 

14-10-199/ 




M7 


^/olob 


A 
A 


07 fXA 1 ArtO 

Z/-04-199O 




wU 


yoiDo^y 


A 

A 


AO AC 1 Arte 

Uo-0o-199d 




1.1 rt 
WU 


QCI CACQ 


A 

A 


AO AC 1 AAC 

Oo— 0o-199b 




AT 
A 1 


1 77977 

lite// 


1 


1 C no 1 AAA 

lb—Oo-1999 




AT 
A 1 


I /D04U 


1 


1 C AO 1 AOA 

lb— 0 J- 1999 




AT 
A 1 


1 onnc 


1 


1 C AC OAAA 

Ib-Ob-ZuOO 




AT 
A 1 


lyuioU 


T 
1 


1 C AO OAAA 

lb— Oj— ZOOO 




AT 
A 1 


iojjb^ 


T 
1 


1 C AO 1 AAA 

lb— O0-1999 




AT 
A 1 


1 7Afi^ 1 


T 
1 


1 C no 1 AAA 

lb— O0-1999 




All 
AU 


71 CfCQO 

/ I boo J 


D 
D 


1 A AO OAAA 

10-02-2000 




All 
AU 


/iy|/irt707 


A 
A 


OA A 1 1 AAO 

Z9-0 1-1998 




All 
AU 


7101 C7 


Q 

D 


oo 1 A 1 AAA 

Z8-10-1999 




Alt 
AU 


yf C'30CQ7 

4bo^by/ 


A 

A 


AC AO 1 AAO 

Ob-OZ-1998 




All 
AU 


oyj/ / b 


0 
D 


AA m 1 nno 
09-07-1998 




All 

AU 


b/ j^yy4 


A 

A 


Ail AT 1 AAil 

04-07-1994 




AM 
AU 


oy^*f£/ 


D 


1 1 AC 1 ArtO 

11-00-1998 




All 
AU 


b/ JoUy** 


A 

A 


nA A 7 lAAil 

04— 0/- 1994 




Alt 
AU 


>! 7Q 

oyi4/y 


D 
D 


01 AC 1 ArtO 

21-0b-1998 




All 
AU 


C7in 
b/ joiy*i 


A 

A 


fkA A7 1 Aa>l 

04— 0/-1994 




All 
AU 


oy^4^o 


D 


1 1 AC 1 AAO 

11-00-1998 




All 
AU 


C7'l19Q/i 

b/ Jo^y*i 


A 

A 


A^ A 7 1An>l 

04— 0/- 1994 




All 
AU 


b/ oOoy*! 


A 

A 


f\A A 7 lAAil 

04— 0/- 1994 




All 
AU 


co>i con>i 
bo'lboy*! 


A 

A 


00 AC 1 rtrtii 

22-06-1994 




All 
AU 


booyoy** 


A 

A 


A>l A7 1 aOA 

04— 0/-1994 




All 
AU 


71 CI 


D 
D 


0>l AO OAAA 

Z4— OZ— ZUOU 




All 
AU 


DUDO/yo 


A 
A 


04-00—1998 




All 
AU 


71 £1 00 


D 
D 


OA AO OAAA 

Z4— OZ-ZOOO 




AU 


6066898 


A 


04-06-1998 




BR 


9307619 


A 


15-06-1999 




BR 


9307620 


A 


10-08-1999 




BR 


9307621 


A 


15-06-1999 




BR 


9307622 


A 


15-06-1999 




BR 


9307623 


A 


16-05-2000 




BR 


9307624 


A 


15-06-1999 



Fomi PCT/ISAOlO (patent tamiy afmex) (July 1 992) 



BNSOCXID: <WO 0T1&449A1 I > 



page 1 of 2 



INTERNATIONAL SEARCH REPORT 

Infarmatkin on patent family members 



Int .tlonal Applicatton No 

PCT/IB 00/01131 



Patent document 
cited in search report 



Publication 
date 



Patent family 
mefnber(s) 



Publication 
date 



US 5798785 



BR 
CA 
CA 



9307625 A 

2151456 A 

2151457 A 



31-08-1999 
23-06-1994 
23-06-1994 



Form PCT/ISA/210 <p3t»ni tamily annax) (July 1 992) 
BNSDOCI[><WO 0116449A1 I > 



page 2 of 2 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 



Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SffiES 



□ BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

" □ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



BEST AVAILABLE IMAGES 




ADED TEXT OR DRAWING 



